Skip to content

feat: add ready-for-dev issue and PR readiness gates - #358

Draft
neubig wants to merge 2 commits into
mainfrom
feat/ready-for-dev-gates
Draft

feat: add ready-for-dev issue and PR readiness gates#358
neubig wants to merge 2 commits into
mainfrom
feat/ready-for-dev-gates

Conversation

@neubig

@neubig neubig commented Aug 24, 2026

Copy link
Copy Markdown
Member
  • A human has tested these changes.

Why

Issue #357 asks for a ready-for-dev issue workflow with CI enforcement, modeled on the equivalent automation in OpenHands/OpenHands and OpenHands/software-agent-sdk but repo-appropriate for this TypeScript client (Node scripts instead of Python, Jest tests instead of pytest, npm-ecosystem reproduction commands).

Summary

  • Issue readiness gate (.github/workflows/issue-readiness-check.yml + .github/scripts/check-issue-readiness.mjs): evaluates bug/enhancement issues against type-specific criteria. Bugs need a reproducible JavaScript/TypeScript command (npm, pnpm, yarn, or npx) in ### Actual Behavior plus an Acceptance Criteria checklist item; enhancements need ### Desired Behavior plus the checklist. Adds/removes the ready-for-dev label idempotently.
  • Pipefail-safe not-ready behavior: in --json mode the readiness script always exits 0 and the workflow branches on the JSON, so a not-ready issue never fails the workflow under set -euo pipefail.
  • Idempotent feedback comments (.github/scripts/post-readiness-comment.mjs): a single comment upserted by a hidden <!-- issue-readiness-check --> marker, posted on open/reopen and on label transitions only.
  • PR gate for linked issues (.github/workflows/pr-description-check.yml + .github/scripts/check-pr-description.mjs): pull_request_target check that fails unless every linked issue (Fixes #N, or bare #N in ## Issue Number) exists and carries ready-for-dev; issues created before the rollout date (2026-08-25) are grandfathered so existing PRs are not retroactively blocked. Bot PRs (dependabot, release-please) are exempt.
  • Label transitions refresh linked PR checks (.github/scripts/refresh-linked-pr-checks.mjs): when ready-for-dev is added/removed on an issue, the latest PR Description Check run is re-triggered for every open PR that links it (via cross-reference timeline events), so the gate never goes stale.
  • Issue forms (.github/ISSUE_TEMPLATE/bug_report.yml, feature_request.yml) so the sections the checker evaluates actually exist for reporters.
  • Tests (src/__tests__/issue-readiness.test.ts, src/__tests__/pr-description-check.test.ts): 35 Jest tests running the real script CLIs in child processes, including the --json exit-0 contract and the linked-issue gate against a local HTTP server standing in for the GitHub API (no mocks).

Issue Number

Fixes #357

How to Test

  • npm run lint && npm run build && npm run test:coverage && npm run format:check all pass locally (343 tests, 20 suites).
  • The new tests exercise the scripts end-to-end: npx jest src/__tests__/issue-readiness.test.ts src/__tests__/pr-description-check.test.ts.
  • After merge: open a test issue via the bug form with/without an npm-style repro command and watch the ready-for-dev label and feedback comment; open a PR linking a non-ready issue and confirm the PR Description Check fails, then passes once the issue becomes ready (label change re-runs the check).

Note: READY_FOR_DEV_ROLLOUT_ISO in check-pr-description.mjs is set to 2026-08-25 (day after authoring). If this PR lands later, bump the constant to the day after merge.

Video/Screenshots

N/A — CI/workflow-only change.

Type

  • Bug fix
  • Feature
  • Refactor
  • Breaking change
  • Docs / chore

Notes

  • The ready-for-dev label already exists in this repo.
  • The new pull_request_target workflow only checks out the base branch SHA (trusted code) and never PR code, matching the security pattern used by the existing pr.yml.
  • This PR was created by an AI agent (OpenHands) on behalf of the user.

@neubig can click here to continue refining the PR

Add an issue-readiness workflow that manages the ready-for-dev label with
type-specific criteria tailored to the TypeScript client: bug reports need a
reproducible JavaScript/TypeScript command (npm, pnpm, yarn, or npx) in the
Actual Behavior section, and both bugs and enhancements need Acceptance
Criteria checklist items (enhancements also need Desired Behavior).

- check-issue-readiness.mjs always exits 0 in --json mode so not-ready
  results drive label/comment behavior without failing the workflow under
  set -euo pipefail
- post-readiness-comment.mjs upserts a single feedback comment via a hidden
  marker (idempotent)
- pr-description-check.yml gates PRs on linked issues carrying ready-for-dev
  (issues predating the rollout are grandfathered)
- refresh-linked-pr-checks.mjs re-runs the PR gate for open PRs linked to an
  issue whose ready-for-dev label changed
- Issue forms for bug reports and feature requests produce the sections the
  readiness check evaluates
- Jest tests exercise the real CLI code paths via child processes, with a
  local HTTP server standing in for the GitHub API

Fixes #357

Co-authored-by: openhands <openhands@all-hands.dev>
@github-actions github-actions Bot added the type: feat A new feature label Aug 24, 2026
@github-actions

github-actions Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Endpoint audit

⚠️ 18 actionable Agent Server contract divergence(s) · report-only

Contract: pinned release artifact

Category Count
Actionable client-only calls 0
Actionable server-only operations 18
Documented non-divergences 13
Agent Server contract operations 129
Audited handwritten client endpoints 119

Actionable client-only calls (0)

none

Actionable server-only operations (18)

  • DELETE /api/canvas-extensions/installed/{}
  • DELETE /api/llm/provider-connections/{}
  • GET /api/canvas-extensions/installed
  • GET /api/canvas-extensions/installed/{}
  • GET /api/canvas-extensions/installed/{}/bundle
  • GET /api/conversations/{}/events
  • GET /api/file/archive
  • GET /api/git/commits
  • GET /api/git/commits/{}/changes
  • GET /api/init
  • GET /api/llm/provider-connections
  • PATCH /api/canvas-extensions/installed/{}
  • PATCH /api/llm/provider-connections/{}
  • POST /api/canvas-extensions/install
  • POST /api/conversations/{}/load_plugin
  • POST /api/file/create_directory
  • POST /api/init
  • POST /api/llm/provider-connections
Documented non-divergences (13)

Client calls intentionally absent from the filtered contract (11)

  • GET /
  • GET /alive
  • GET /health
  • GET /ready
  • GET /server_info

Reason: Operational Agent Server endpoints intentionally excluded from the filtered public release artifact.
Owner: OpenHands runtime maintainers

  • DELETE /api/meta-profiles/{}
  • GET /api/meta-profiles
  • GET /api/meta-profiles/{}
  • POST /api/meta-profiles/{}
  • POST /api/meta-profiles/{}/activate

Reason: Client-ahead API stacked on the pending Agent Server meta-profiles implementation.
Owner: OpenHands SDK maintainers
Tracking: OpenHands/software-agent-sdk#3744

  • POST /api/profiles/{}/validate

Reason: Client-ahead API stacked on the pending Agent Server pre-flight LLM validation endpoint.
Owner: OpenHands TypeScript client maintainers
Tracking: OpenHands/software-agent-sdk#4422

Server operations covered by an exposed browser URL (2)

  • GET /api/conversations/{}/workspace
  • GET /api/conversations/{}/workspace/{}

Reason: RemoteWorkspace.startWorkspaceSession exposes these authenticated URLs for browser iframe and file requests; they are not HttpClient method calls.
Owner: OpenHands TypeScript client maintainers

@neubig
neubig requested a review from all-hands-bot August 29, 2026 12:07
@all-hands-bot

Copy link
Copy Markdown
Contributor

🤖 OpenHands is reviewing this PR.

Head commit: 7d7dfdf5c1136e5191f155ede42c40e44dad0c0d
View the conversation: https://oss-agent-canvas.ngrok.dev/conversations/182ce02a-fb22-4e08-86a5-6cded6f6b42f

This comment was posted by an AI agent (OpenHands).

@all-hands-bot all-hands-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This review was created by an AI agent (OpenHands) on behalf of the repository maintainers.

Verdict

Approve with two minor follow-ups. This is a well-structured, security-conscious CI-only change that correctly ports the ready-for-dev gate to the TypeScript ecosystem (Node scripts, Jest, pull_request_target checking only the base SHA). The pipefail-safe --json exit-0 contract, the hidden-marker comment upsert, and the grandfathering for pre-rollout issues are all thoughtful touches. No material bugs or security problems found.

Risk assessment

Low. No runtime or library code changes (src/ only gains tests). The risk is operational: the gate may behave more permissively than documented around missing/typo'd issue references, and the rollout date constant is already stale.

Two inline comments below are worth addressing before this ships, but neither is blocking.

Comment thread .github/scripts/check-pr-description.mjs Outdated
Comment thread .github/scripts/check-pr-description.mjs Outdated
Co-authored-by: openhands <openhands@all-hands.dev>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type: feat A new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add ready-for-dev issue and PR readiness gates

3 participants